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I.  INTRODUCTION 

i 

—yThm  proposed  BASIC  program  -for  smoothing  NUWES  data  uses  the  Least- 
Squares  Method  to  'fit  a  polynomial  to  a  7-point  data  segment.  It  determines 
the  best  fitting  polynomial  of  order  1,  2,  or  3  and  establishes  an  estimate 
for  the  coordinate  at  the  time  in  the  center  of  the  data  segment.  It  also 
establishes  a  Figure-of -Merit  (FM)  for  that  estimate. 

This  program  is  designed  specifically  for  treatment  of  outliers  and 
missing  points  but  it  can  also  be  used  for  smoothing  of  other  observations. 
It  is  intended  to  be  used  as  the  basic  smoothing  program  for  NUUIES  data  and 
to  provide  a  value  of  FM  for  each  observation  time  for  feedback  to  the 
Instrumentation  Department.  Values  of  the  FM's  also  provide  information 
for  other  users  of  the  smoothed  data  on  it's  quality  (i.e. ,  hoM  well  the 
smoothed  values  can  be  expected  to  represent  the  actual  path  of  a  vehicle^)* 

This  report  is  intended  to  supplement  the  Project  Report  entitled 
"A  Figure  of  Merit  for  NUWES  Data".  (Ref.  7) 


II.  DATA  PREPARATION 

It  is  assumed  that  the  input  data  has  been  listed  in  the  following 
format 


Time,  Array,  x  comp,  y  comp,  z  comp. 

Each  component  <x,y,2)  is  smoothed  separately  and  the  Array  identification 
is  not  used  in  the  smoothing  process  so  a  separate  file  can  be  constructed 
for  each  component  with  the  Array  column  replaced  by  a  column  for 
identification  of  Questionable  Points  <QP) ,  i.e.,  for  identification  of 
Missing  Paints  and  Outliers.  The  identification  numbers  are  specified  with 
Array  A<6,5)  which  is  presented  in  the  next  section  of  this  report.  The 
proposed  input  format  is 

Time,  QP,  XO 

where  XO  is  the  observed  value  of  x  at  the  corresponding  observation  time. 
Two  preliminary  steps  must  be  taken  before  the  smoothing  can  be  performed. 
These  are: 


A.  Treatment  of  Missing  Points.  Since  the  program  is  designed  to  treat 
'-point  data  segments  containing  no  missing  points,  the  data  must  be 
screened  and  temporary  values  provided  at  each  missing  point.  These 
temporary  values  are  established  by  taking  a  linear  average  of  adjacent- 
observational  values.  The  following  formulae  are  used  to  supply  the 
temporary  values: 


□ 

□ 


1.  For  a  single  missing  value  at  time  t  when  there  are 
observed  values  :!o(t-l)  and  xo(t-t-l)  the  temporary  value  is 

^y. 

;;o<t)  =  (xo(t-l)  +  xo(t-*-l )  ) /2. 


Oi^tibution/ 
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2.  For  adjacent  missing  values  xa(t>  and  xo(t'4-l)  when  there  are 
observed  values  xo<t-l>  and  xo(t+2)  the  temporary  values  are 

xo<t)  “  <2*xo<t-l)  +  xo<t+2> ) /3, 
xo(t+l)  »  (xo(t-l)  +  2*xo<t+2> ) /3. 

3.  For  three  adjacent  missing  values  xo(t-l),  xo<t),  and  xo(t-t-l> 
when  there  are  observed  values  xo(t-2)  and  xo(t4-2)  the  temporary  values 


xo<t-l)  »  (3*xo<t-2)  +  xo(t-«-2)  ) /4, 
xo<t)  »  <XQ<t-2)  +  xo(t+2))/2, 

!<o<t  +  l)  =  <xo<t-2)  +  3'»xo  (t+2)  ) /4. 

4.  For  -four  adjacent  missing  values  xo(t-l>,  xo(t), 
xo(t+l),  and  xo<t+2)  when  there  are  observations  xo(t-2)  and  xo(t+3) 
the  temporary  values  are 

xo<t-l)  =  (4*xo(t-2)  +  xo(t+3>)/5, 
xa<t)  =  (3*xo(t-2)  +  2*xo (t+3) ) /5 , 
xo(t+l)  =  (2*xo(t— 2)  +  3*xo (t+3> > /5 , 
xo(t+2)  =  (xo<t-2)  +  4*xo(t+3) ) /5. 

5.  When  there  are  more  than  -four  adjacent  missing  values  in  a  7- 
point  data  segment  the  Least-Squares  Method  cannot  be  used  to  establish 
an  estimate  ■far  the  midpoint  o-f  the  segment.  (It  could  still  be 
possible,  in  some  cases,  to  smooth  at  other  points  than  the  midpoint  of 
the  segment.  This  has  not  been  considered  as  it  would  require 
considerable  extension  o-f  the  smoothing  program  and  its  desirability 
would  be  questionable.) 

B.  Ident  i -f  i  cat  i  on  o-f  Outliers.  The  procedure  for  identification  of 
Outliers  involves  the  use  of  Sequential  Differences  (Ref. 2  and  Ref. 5).  It 
will  not  be  discussed  in  detail  here.  A  Fourth  Order  difference  of 
magnitude  50  or  greater  is  considered  an  indicator  that  the  corresponding 
value  of  xo  is  a  potential  outlier.  When  there  are  adjacent  potential 
outliers,  the  one  with  the  greatest  magnitude  of  Fourth  Order  difference  is 
identified  as  the  actual  outlier  and  assigned  the  appropriate  number  in  the 
second  column  of  Array  A(6,5).  After  smoothing  this  outlier,  it  might  be 
prudent  to  recalculate  the  Sequential  Differences  with  the  outlier  replaced 
by  its  smoothed  value  xe<t)  to  give  assurance  that  the  adjacent  potential 
outliers  are  no  longer  identified  as  such.  Cases  where  adjacent  potential 
outliers  will  also  be  identified  as  outliers  are,  hopefully,  rare  and  should 
be  flagged  for  special  treatment. 


III.  THE  SMOOTHING  PROGRAM 


A.  ARRAYS 

The  program  presented  in  this  section  was  written  in  the  BASIC  Language 
for  use  on  an  IBM  PC.  It  uses  6  Arrays  which  will  be  discussed  in  the  order 
that  they  appear  in  the  program. 


irst 

Array 

to  be 

used  is  A <6 

,5).  It 

is  shown 

below 

ARRAY 

A(6,5) 

Aj 

0 

1 

2 

3 

4 

5 

T 

QP 

XQ 

XM 

XE 

RES 

0 

—3 

(0) 

xo(-3) 

xm (-3) 

xe (-3) 

r  (-3) 

1 

-2 

(-1) 

xo(-2) 

xm (-2) 

xe (-2) 

r  (-2) 

2 

-1 

(-2) 

xo (— 1 ) 

xm (-1 ) 

xe (-1 ) 

r  (-1) 

3 

0 

(-10) 

xo  (0) 

xm  (0) 

xe  (0) 

r  (0) 

4 

+  1 

(-11) 

xo (+1 ) 

xm (+1 ) 

xe (+l ) 

r  (  +  1) 

5 

+2 

(-12) 

xo(+2) 

xm (+2) 

xe (+2) 

r  (+2) 

6 

+3 

- 

xo (+3) 

xm (+3) 

X  e ( +3 ) 

r  (+3) 

QUESTIONABLE  POINTS  (QP) 

0  >io  is  legitimate  observed  value 
-1  xo  is  unscheduled  missing  point 
-2  xo  is  scheduled  missing  point 
-10  xo  is  identified  as  an  outlier 

-11  xo  is  an  unscheduled  missing  point  and  an  outlier 
-12  xo  is  a  scheduled  missing  point  and  an  outlier 

Smoothing  of  a  7-paint  data  segment  is  performed  to  establish  the 
estimate  and  FM  for  the  midpoint  of  the  segment.  The  times  in  the  first 
column  of  Array  A  is  obtained  by  subtracting  the  time  of  this  midpoint  from 
the  observation  times  of  the  data  segment  so  that  column  1  of  this  array 
will  always  be  the  same  for  all  data  segments.  The  second  column  of  Array 
A  provides  the  identification  of  the  QP's.  A  separate  identification  for 
scheduled  missing  paints  may  be  omitted  but  is  included  for  potential 
future  use.  The  XO's  (column  3)  are  the  input  observed  values  including 
the  temporary  values  for  the  missing  points.  The  other  columns  of  this 
array  will  be  discussed  as  they  appear  in  the  smoothing  program. 

Array  T (5)  provides  the  table  values  of  the  Student-T  distribution 
that  are  needed  to  establish  the  Figure-of -Merit  (FM)  which  accompanies  the 
estimate  xe  at  the  midpoint  of  the  data  segment.  As  discussed  in  Reference 
7,  the  number  infinity'  cannot  be  used  by  a  computer  and  hence  it  is 
arbitrarily  replaced  by  the  number  99.99  for  T(0) .  This  value  is  large 
enough  to  produce  a  large  value  of  FM  and  hence  to  indicate  low  quality  of 
the  estimate  it  accompanies.  The  argument  j  in  T(j)  ranges  from  0  to  5  and 
is  the  appropriate  number  of  ' degrees-of -freedom '  produced  in  Array  B(2,3). 
This  will  be  discussed  later. 


Array  T (5)  is  shown  bslow 


ARRAY  T(5) 

j 

T(  j> 

0 

99.99 

1  2 
6.314  2.920 

3 

2.353 

4 

2. 132 

5 

2. 105 

ARRAY  1 

B<2,3> 

i  \  j 

0 

1 

2 

3 

K 

DFK 

SDRK 

FMK 

0 

1 

DFl 

SDRl 

FMl 

1 

2 

DF2 

SDR2 

FM2 

2 

3 

DF3 

SDR3 

FM3 

In  this  array  K  is 

the 

order  of 

the  polynomial 

being 

considered  (k=l  for 

linear,  k=2  for  quadratic. 

and 

k=»3  for 

cubic ) . 

DFK  is 

the  'degrees  of 

freedom'  available 

for 

fitting 

a  polynomial  of 

degree 

k  and  is  establishe 

by  ^^irst  determining  the  number  NS  (element  C(3>  in  Array  C(S>)  and  then 
reducing  this  number  by  the  number  o-f  parameters  (k-*-l)  in  the  polynomial. 
Thus 


NS  =  N  -  (number  o-f  QP's  in  column  A(l,j)> 

“  C(3)  -  (number  o-f  negative  entries  in  column 

A(l,  j)  ) 


with,  initially, 

C(3)  =  7  =  number  o-f  points  in  the  data  segment. 
Columns  2  and  3  are 


B(i,2>  ®  estimate  of  the  standard  deviation  of  the  noise 
in  the  observations  (the  xo's)  when  a  polynomial 
of  order  k  is  fitted  to  the  data  segment 

and 

B(i,3)  =  the  Figure-of-Merit  for  the  estimate  xeCO). 

ARRAY  S<5) 


j  0  1  2  3  4  5 

S(j)  xav  ssx  six  s2x  s3x  s3x 1 

The  element  S(0)=>xav  is  the  average  of  the  xo's.  It  is  needed  to 
produce  the  xm's  (column  3  in  Array  A).  The  other  elements  of  S  are  used  to 
calculate  the  values  for  the  SDRK ' s  and  the  FMK ' s  in  Array  B.  The  elements 
of  this  array  must  be  zeroed  at  the  start  of  each  smoothing  cycle  (line  295 
of  bhe  program)  since  they  contain  cumulations  of  product  of  the  data. 


ARRAY  D<3) 


j  0  1  2  3 

D(J)  bkO  bkl  bk2  bk3 

This  array  is  for  ths  cosfficionts  of  ths  polynomial  with  tha  ordsr  k 
sslectsd  to  fit  the  data  segment.  Its  elements  must  be  zeroed  before  each 
iteration. 

ARRAY  C(8) 

j0123  456  78 

C(j)  T  QP  XO  NS  ITER  K  XE  RES  FM 

This  array  provides  the  final  output  of  the  smoothing  program.  In 
addition  to  the  time  <C<0))  for  which  the  smoothing  was  performed  and  the 
observed  or  temporary  value  of  mo  (C<2))  at  that  time,  it  identifies  the 
nature  of  the  QP  (C(l)>  and  the  number  of  legitimate  observations  in  tahe 
data  segment.  It  reports  the  number  of  iterations  of  the  smoothing  process 
(C(4))  and  the  order  of  the  polynomial  used  to  fit  the  data.  It  contains 
the  smoothed  value  XE  <C(6))  and  the  accompanying  value  of  FM(C(8)). 

Finally,  it  includes  the  residual  error  <C(7>>  which  is  the  difference 
between  the  original  value  of  >:a  (C(2))  and  the  estimated  value  (C(6>). 

Notfj  that  this  residual  error  can  differ  from  the  value  of  the  residual 
error  r<0)  in  cell  A(3,5)  of  Array  A  when  smoothing  is  Iterated.  For 
example,  if  A(3,l)  (and  C<1))  has  the  value  -10,  then  the  residual  in  C(7) 
is  the  difference  between  the  outlier  value  of  xo  in  C(2)  and  the  estimated 
value  xe  in  C(6)  whereas  the  residual  error  in  A(3,5>  is  the  difference 
between  the  estimated  value  xe<0)  in  cell  A<3,4>  after  the  last  iteration 
and  the  value  of  xo(0>  in  cell  A(3,2)  which  is  the  value  of  xe(0>  in  cell 
A(3,4)  from  the  preceding  iteration. 


.■^'  j'4  , 
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B.  THE  PROGRAM 

A  listing  of  the  BASIC  smoothing  program  follows. 

BASIC  PROGRAM  LS7T3.DOC 


05  REM  LS7T3.DOC 


17  Feb. , 1985 


2.353,  2.132,  2.105 


(QP's) 


10  DIM  A(6,5),  B(2,3),  C(8),  D(3),  D(5>,  T(5) 

20  DATA  -3,  -2,  -1,  0,  1,  2,  3 
30  FOR  i=0  TO  6 
40  READ  A(i ,0) 

50  NEXT  i 
55  REM  STUDENT  T 

60  DATA  99,99,  6.314,  2.920,  2.353,  2.132,  2. 1C 
70  FOR  j=0  TO  5 
80  READ  T(j) 

90  NEXT  j 
100  C(3)  =7 

105  REM  ENTER  SMOOTHING  TIME 
110  C(0)  “  2160 

115  REM  ENTER  QUESTIONABLE  POINTS  (QP's) 

120  DATA  -10,  0,  0,  -2,  0,  0,  0 
130  FOR  i=0  TO  6 
140  READ  A<i ,1) 

150  NEXT  i 

160  C<1)  =•  A<3,1) 

165  REM  ENTER  OBSERVED  XO 

170  DATA  33238,6,  33267.7,  33313.5,  33374.2, 

33592.5 

180  FOR  i=0  TO  6 
190  READ  A<i ,2) 

200  NEXT  i 

210  C(2)  =  A(3,2) 

215  REM  DETERMINE  NS 
220  FOR  i=0  TO  6 

230  IF  A<i,l)<0  THEN  C(3)=C<3)-1  ELSE  C(3)=C(3> 
240  NEXT  i 

245  REM  CHECK  IF  SMOOTHING  POSSIBLE 

250  IF  C(3)<3  THEN  GO  TO  260  ELSE  GO  TO  290 

260  C<4)=0  ;  C(5)=0  C(6)=C(2) 

270  C(7>=0  !  C(a)=T(0) 

230  GO  TO  1050 
290  C<4)=0 

295  REM  START  SMOOTHING 

300  C<4)=C(4>+1 

305  REM  ZERO  S  AND  D  CELLS 

310  FOR  j=0  TO  5 

320  S ( j ) =0 

330  NEXT  j 

340  FOR  j=0  TO  3 

350  D ( j  >  =0 

360  NEXT  j 

363  REM  PRELIMINARY  CALCULATIONS 
366  REM  CALCULATE  XAV  AND  XM ' s 
370  FOR  i=0  TO  6 


33313.5,  33374.2,  33434.8,  33510.5, 


•  ITERATION 


.V.V.-.-. 


*  ^ 


.N  .V. 

r\-'> 


'X* .  • . » 


■T- -f- 


6 


380  S<0)»S<0)+A<i ,2) 

390  NEXT  i 

400  S<0)»S(0)/7  'XAV 

410  FOR  i«0  TO  6 

420  A(i ,3)»A(i ,2)-S(0)  ' XM 

430  NEXT  i 

435  REM  CALCULATE  SUMS  OF  PRODUCTS 
440  FOR  i»0  TO  6 

450  S(l)=S(l)+A(i ,3)*A(i ,0)  'SSX 

460  S(2)-S(2>+A(i ,3>*A(i ,0)  'SIX 

470  S(3)-S(3)+A<i ,3)*(A<i ,0)^2)  'S2X 

480  S(4)=S<4)+A(i  ,3)*<A(i  ,0) '•3)  'S3X 

490  NEXT  i 

500  S<5)»(28*S(4)-196*S(2) ) /28  'S3X1 

505  REM  CALCULATE  FMl 

510  B(0,l)»C(3>-2  'DFl 

520  SSRl=(2a*S<l)-S<2) ''2) /2a 

530  B(0,2)=SQR(SSR1/B<0, 1) )  'SDRl 

540  B<0,3)=B<0,2)*T(B<0, 1) ) /SQR<C(3) )  ‘FMl 

545  REM  CHECK  QUADRATIC  FIT 

550  B<l,l)=C<3)-3  'DF2 

560  IF  B(  1,1X1  THEN  GO  TO  690  ELSE  GO  TO  570 

565  REM  CALCULATE  FM2 
570  SSR2=<a4*SSRl-S(3)-''2> /a4 

580  B(l,2>=SaR<SSR2/B(l,l) )  'SDR2 

590  B<1,3)=B<1,2)*T<B<1,1) )/SQR<C<3)  'FM2 

595  REM  CHECK  CUBIC  FIT 

600  B(2, l)«C<3>-4  'DF3 


610  IF  B<2,1)<1  THEN  GO  TO  620  ELSE  GO  TO  640 
620  B<2,2)=0  :  B(2,3)=T(0) 

630  GO  TO  670 

635  REM  CALCULATE  FM3 

640  SSR3=<216*SSR2-S<5)  ''2) /216 

650  B(2,2)=SaR<SSR3/B(2, 1) )  'SDR3 

660  B(2,3)=B<2,2)*T<B<2, 1) ) /SQR(C<3) )  'FM3 

665  REM  SELECT  POLYNOMIAL 

670  IF  B(2,3)<B(1,3)  AND  B < 2 , 3) <B <0 , 3)  THEN  GO  TO  850 
ELSE  GO  TO  680 

680  IF  B(1,3XB(0,3)  THEN  GO  TO  770  ELSE  GO  TO  690 
685  REM  FIT  LINEAR 

690  C(5)=l  ;  C(8)=B<0,3)  'k=l  FM=FM1 

69S  REM  ESTABLISH  COEFFICIENTS 

700  D(1)=S(2) /28  'b21 

705  REM  DETERMINE  XE,  RES 
710  FOR  i=0  TO  6 

720  A(i ,4)=S<0)+D(l)*A(i ,0)  'we 

730  A ( i , 5) =A ( i , 2) — A < i , 4 )  'res 

740  NEXT  i 

750  C(6)=A(3,4)  ;  C (7) =C (2) -C (6) 

760  GO  TO  930 
765  REM  FIT  QUADRATIC 
770  C(5)=2  !  C(8)=B(1,3) 


k=2  FM=FM2 


775  REM  ESTABLISH  COEFFICIENTS 

780  D<2)=S(3)/a4  i  D<1)=  S(2)/28  s  0(0)— 4*D(2) 

785  REM  DETERMINE  XE,  RES 
790  FOR  i=0  TO  6 

800  A(i  ,4)=S(0)+D(0>+D<l)*A(i  ,0)+D(2)*(A<i  ,0>''2)  xe 

810  A ( i ,5) *A (i ,2) -A (i , 4)  'res 

820  NEXT  i 

830  C(6)=A(3,4)  :  C <7) «C (2) -C (6) 

840  GO  TO  930 
845  REM  FIT  CUBIC 

850  C<5)=3  :  C(8>=B<2,3)  'k=3  FM=FM3 

855  REM  ESTABLISH  COEFFICIENTS 

860  D(3)=S(5) /216  :  D<2)=S<3)/a4 

870  D(1)=(S<2)-196*D<3) ) /2a  :  D(0)*-4*D(2) 

875  REM  DETERMINE  XE  RES 
380  FOR  i«0  TO  6 

890  A(i  ,4)=S<0)+D(0)+D(l)*A<i  ,0)-*-D(2)*(A(i  ,0)-'2)+D<3)*(A<i  ,0)'3) 
900  A<i ,5)=A<i ,2)-A<i ,4)  'res 

910  NEXT  i 

920  C(6)=A(3,4)  :  C (7) =C (2) -C <6) 

925  REM  TERMINATE  IF  NOT  OP 

930  IF  A<3,1)=0  THEN  GO  TO  1050  ELSE  GO  TO  940 
935  REM  CHECK  RESIDUALS 
940  FOR  iaO  TO  6 

950  IF  A(l,i)<0  AND  ABS < A ( i , 5) ) > 1  THEN  GO  TO  980  ELSE  GO  TO  960 
960  NEXT  i 
970  GO  TO  1050 

980  IF  C<4)>9  THEN  GO  TO  1050  ELSE  GO  TO  990 
990  FOR  i=0  TO  6 

1000  IF  A(i,l)<0  THEN  A ( i , 2) =A ( i , 4 )  ELSE  A ( i , 2) =A ( i , 2) 

1010  NEXT  i 
1020  GO  TO  300 
1045  REM  TERMINATE 
1050  FOR  j=0  TO  8 
1060  LPRINT  C(j); 

1070  NEXT  j 
1080  LPRINT 
1090  STOP 


C.  DISCUSSION  OF  THE  PROGRAM 

The  smoothing  program  can  now  be  discussed  in  some  detail. 

STEP  1.  Enter  the  segment  data. 

Be-fore  starting  the  smoothing  the  pertinent  information  on  the  data 
segment  must  be  entered.  This  includes  the  time  (program  line  110),  the 
QP's  (line  120),  and  the  observed  xo's  (line  170). 

STEP  2.  Determine  NS. 

This  is  the  number  of  legitimate  observations  in  the  segment. 

(1 ines220-240) 

STEP  3,  Check  if  smoothing  possible. 

If  NS<3,  then  even  linear  polynomial  cannot  be  fitted  to  the  data 
segment  by  the  Least-Squares  Method.  (lines  250-280)  If  no  fit  is 
passible,  then  terminate  (STEP  15). 

STEP  4.  Zero  iteration  counter  and  cells  in  Arrays  S  and  D. 

S  cells  cumulate  data  products  and  D  cells  contain  parameter 
coefficients,  (lines  290-360) 

STEP  5.  Calculate  xav  (S(0))  and  xm's  (A(i,3). 

xav  is  the  average  of  the  xo's  and  the  xm's  are  the  differences 
xm ( j ) =xo ( j ) -xav.  This  reduces  computational  round-off  error. 

(lines  370-430) 

STEP  6.  Calculate  other  S(j) 's. 

These  are  sums  of  products  of  data  values,  (lines  440-500) 

STEP  7.  Determine  capability  of  linear  fit. 

Establish  DPI  (B(0,1)),  SDRl  (B(0,2)),  and  FMl  (B(0,3)). 

(lines  510-540) 

STEP  8.  Check  quadratic  fit. 

If  NS  <  4  then  DF2  <  1  and  no  quadratic  fit  is  possible.  Otherwise 
establish  SDR2  and  FM2.  (lines  550-590) 

STEP  9.  Check  cubic  fit. 

If  NS  <  5  then  DF3  <  1  and  no  cubic  fit  is  possible.  Otherwise 
establish  SDR3  and  FM3.  (lines  600-660) 


STEP  10.  Select  polynomial  for  fitting  data  segment. 


I-f  Fli3  <  FM2  and  FM3  <  FMl  then  select  cubic,  set  k  =  C(5)  =  3  and  fit 
cubic  in  STEP  13.  Otherwise,  if  FM2  <  FMl  then  select  quadratic,  set 
k  =  C(5)  =  2  and  fit  quadratic  in  STEP  12.  Finally,  if  neither  of  the  above 
then  select  linear,  set  k  =>  C(5)  »  1  and  fit  linear  in  STEP  11. 

(lines  670-680). 

STEP  11.  Fit  linear  polynomial. 

Establish  linear  coefficients  (D(i)'s>  and  calculate  i<e(i)'s  and 
r<i)'s,  also  xe(0)  in  cell  C<6)  and  res  in  cell  C(7).  (lines  690-750) 

STEP  12.  Fit  cubic  polynomial. 

Establish  quadratic  coefficients  and  calculate  estimates  and  residuals, 
(lines  770-830) 

STEP  13.  Fit  cubic  polynomial. 

Establish  cubic  coefficients  and  calculate  estimates  and  residuals, 
(lines  850-920) 

STEP  14.  Check  QP's. 

If  xo(0)  is  legitimate  observation  (A(3,l)  =  0)  then  terminate  (STEP 
15).  Otherwise  check  number  of  iterations  (C(4)).  If  C(4)  =  10  then 
terminate.  Finally,  check  residuals  at  times  where  A(i,l><0  (at  missing 
paints  and  outliers).  If  any  of  these  are  greater  than  unity  in  magnitude 
then  repeat  the  smoothing  (STEP  4).  If  none  of  them  are  then  terminate, 
(lines  930-1020) 

STEP  15.  Terminate. 

Print  Array  C  and  STOP,  (lines  1050-1090) 


IV.  APPLICATION 


The  example  used  to  illustrate  this  smoothing  program  is  the  sample 
selected  consists  o-f  40  points  from  times  ts2121  to  t=2160.  Additional 
points  at  both  ends  of  the  sample  are  required  in  order  to  smooth  at  these 
times.  Appropriate  inputs  for  this  sample  are  given  in  the  first  four 
columns  of  Table  1.  It  is  convenient  to  have  the  contents  of  Figure  1  on 
the  monitor  when  performing  the  data  smoothing.  Inputs  can  then  be  changed 
easily  by  making  the  appropriate  changes  in  lines  110,  120,  and  170.  This  is 
particularly  convenient  when  smoothing  values  other  than  outliers  or  missing 
values.  The  computer  printout  of  the  results  of  smoothing  this  sample  are 
shown  in  Figure  2  and  summarised  in  the  last  three  columns  of  Table  1. 

It  is  of  some  interest  to  compare  the  results  in  Table  1, 
established  using  the  BASIC  Program  LS7T3.BAS  on  an  IBM  PC,  with  the  results 
given  in  Table  1  of  Reference  7  which  were  established  using  a  Texas 
Instruments  calculator  (TI-59) .  The  differences  can  be  attributed  to  the 
differences  in  the  number  of  places  used  in  the  computations. 

Operation  of  the  smoothing  program  can  be  examined  in  more  detail  by 
revising  the  program  to  version  LS7T4.BAS.  Application  of  this  revised 
program  to  the  treatment  of  the  missing  point  at  t=2160  is  shown  in  Figure 
3a.  This  figure  presents  the  elements  of  the  other  arrays  as  well  as  Array 
C.  Several  aspects  of  the  smoothing  of  this  missing  point  are  discussed 
bel ow. 

1.  The  initial  value  value  for  xo(0)  at  time  t*2160  which  is  entered 
in  A<3,2)  is  the  temporary  value  xo (0) = (xo (-1 ) +xo (+1 ) > /2“33374. 2. 
This  value  also  appears  in  C(2)  of  the  output.  The  value  shown  in 
A<3,2)  in  the  printout  of  Array  A  is  the  value  of  xe(0)  in  A<3,4) 
from  the  next  to  last  iteration. 

2.  The  initial  value  of  xo<— 3)  entered  in  cell  A(0,2)  is  the  smoothed 
value  for  the  outlier  at  t=2157  which  was  established  when  that 
outlier  was  smoothed.  Again,  the  value  in  A(0,2)  in  the  printout 
of  Array  A  is  the  value  of  xe(— 3>  in  A(0,4)  from  the  next  to  last 
iteration. 

3.  The  residual  error  r(0)=0.184  in  A(3,5)  is  the  difference  between 
the  values  of  xo<0)  in  A<3,2)  and  xe(0)  in  A(3,4)  whereas  the 
residual  error  in  C(7)  is  the  difference  between  the  temporary 
value  of  xo<0)  initially  entered  in  A<3,2)  and  in  C(2)  and  the 
smoothed  value  xe<0)  in  A(3,4)  and  in  C(6>. 

4.  In  each  iteration  of  the  smoothing  process  the  value  of  xo(0)  in 
A(3,2)  was  replaced  by  the  estimate  xe(0>  in  A(3,4).  The 
difference  r (0)  shown  in  A<3,5)  is  r (0) =xo (0) -xe (0)  which  is 
displayed  in  Array  A  as  A (3,5) =A<3,2> -A (3,4) =0. 2.  The  residual 
error  shown  in  C(7)  is  the  difference  between  the  origninal  value 
in  A(3,2)  which  is  saved  in  C(2)  and  the  final  estimate  xe(0)  in 
C(6)  and  in  A(3,4)  so  that  it  is  the  difference  between  the  initial 
temporary  value  and  the  final  smoothed  value, 


C<7)=»C<2>-C<6)»4.a 


5.  In  the  same  way,  the  residual  error  in  A(-3,5)  is  the  di-f-ference 
A(-3,5)=A<3,2)-A<3,4)«0. 1 

whereas  the  -fitting  error  at  t*2157  is  the  di-fference  between  the 
Initial  value  in  A(-3,2)  which  is  the  previously  smoothed  value 
(and  also  the  input  value)  and  the  smoothed  value  o-f  xe(-3)  in  A(- 
3,2)  so  that 


RES ( -3 ) =33238 . A-33234 . S=4 . 1 . 


The  final  estimate  xe<-3)  in  A<-3,4)  is  consistent  with  the  7-point  data 
segment  centered  at  t=2160  whereas  the  initial  value  xo(-3)  is  consistent 
with  the  7-point  data  segment  centered  at  t=2157.  This  brings  up  the 
question  of  whether  previously  smoothed  values  should  be  iteratively 
smoothed  when  smoothing  other  points.  This  warrants  further  investigation. 


TREATMENT  OF  MISSING  POINT  AT  T  »  2160 


INPUT  OP'S 

-10  0  0  -2  0  0  0 
INPUT  XO'S 

33238.6  33267.7  33313.5  33374.2  33434.8  33510.5  33592.5 


ARRAY  C  (OUTPUT) 

2160  -2  33374.2  533  33369.4  4.804688  1.941253 


ARRAY  A 
-3  -10 

33234.56 

-154.457 

33234.5 

.0625 

-2 

0 

33267.7 

-121.3203 

33267.94 

-.2421875 

-1 

0 

333 13.5 

-75.51953 

33313.25 

.  25 

0 

-2 

33369. 58 

-19.44141 

33369 . 4 

. 1835938 

1 

0 

33434.8 

45.78125 

-.5507813 

2 

0 

33510.5 

121.4805 

33510. 1 

. 4023438 

3 

0 

33592.5 

203.4805 

33592.6 

-. 1015625 

ARRAY  B 

0  3  26.00435  27.36421 

0  2  1.84462  2.408822 

0  1  .687484  1.941253 


ARRAY  S 

33389.02  102914.5  1680.715  412.1133  11728.02  -36.98438 

ARRAY  D 

-19,62444  61.2241  4.906111  -.171224 


TABLE  1 

Smoothing  Sample  NWS2AX1 


T 

COLL 

xo 

QP 

XE 

RES 

2117 

3 

33533.2 

0 

- 

- 

2118 

3 

33567.7 

0 

— 

— 

2119 

3 

33603 . 6 

0 

— 

— 

2120 

m 

(33634. 1) 

-2 

33634.9 

(-0.8) 

2121 

3 

33664.4 

0 

33665. 3 

-0.7 

2122 

3 

33695.0 

0 

33693.5 

1.5 

2123 

3 

33718.5 

0 

33720.9 

-2.4 

2124 

3 

33745.8 

0 

33744.6 

1.2 

2125 

3 

33767. 1 

0 

33764.6 

2.6 

2126 

3 

33780.7 

0 

33783.7 

—3. 0 

2127 

"S’ 

o 

33798.0 

0 

33799. 7 

-1.7 

2128 

m 

(33810.9) 

33816.3 

(-5.4) 

2129 

3 

33823. 7 

0 

33828. 1 

-4.4 

2130 

3 

33827. 3 

0 

33821.6 

5.7 

2131 

3 

33794.2 

0 

33786.6 

7.6 

2132 

3 

33726. 1 

0 

33724. 1 

2.  1 

2133 

3 

33637 . 7 

0 

33640. 1 

-2.4 

2134 

12 

33556.5 

0 

33556. 9 

-0.4 

2135 

12 

33486 . 6 

0 

33490.0 

-3.4 

2136 

m 

(33466.5) 

-12 

33452.3 

(14.2) 

2137 

•j 

33446.5 

0 

33451.2 

-4.7 

2138 

33485. 1 

0 

33489.9 

-4.8 

2139 

3 

33559.3 

0 

33560. 9 

-1.6 

2140 

3 

33650 . 2 

0 

33649. 8 

0.4 

2141 

3 

33738.5 

0 

•^o7 o4 . 

4.0 

2142 

3 

33799.0 

0 

33795.6 

3.4 

2143 

3 

33825 . 8 

0 

33823. 1 

2.7 

2144 

m 

(33798.7) 

-12 

33813.0 

(-14.3) 

2145 

3 

33771.6 

0 

33767.4 

4.2 

2146 

33698  ■  3 

0 

01^696 . 

1.8 

2147 

33607 . 7 

0 

33614.6 

-6.9 

2148 

12 

33528.5 

0 

o>o.jo0 . 0 

-1.3 

2149 

12 

33455 . 2 

0 

33451.7 

3.5 

2 1 50 

33381.2 

0 

33383. 4 

-2.2 

2151 

3 

33-.i>2o .  5 

0 

33325. 2 

-1.7 

2152 

m 

(33285.  1) 

33277.9 

(7.2) 

2153 

33246.6 

0 

33243.5 

3.  1 

2154 

-r 

33219.5 

0 

33222. 1 

-2.6 

2155 

3 

33212.5 

0 

33214.0 

-1.5 

2156 

oo22 1.0 

0 

33218.8 

2.2 

2157 

33273.5 

-10 

33238.6 

34.9 

2158 

33267.7 

0 

33269.7 

-2.0 

2159 

3 

33313.5 

0 

33313. 1 

0.4 

2160 

m 

(33374.2) 

Am 

33369 . 4 

(4.8) 

2161 

3 

33434. a 

0 

— 

— 

2162 

3 

33510. 5 

0 

- 

— 

2163 

33592.5 

0 

- 

— 

14 


FIGURE  1 

BASIC  SCREEN  FOR  SMOOTHING  DATA 


LPRINT 


LPRINT 


"SAMPLE  NWS2AX1" 


"OUTLIERS" 


LIST  110 
110  C(0)=2160 
0 

LIST  120 

120  DATA  -10,  0,  0,  -2,  0,  0,  0 
0 

LIST  170 

170  DATA  33238.6,  33267.7,  33313.5,  33374.2,  33434.8,  33510.5,  33592.5 


«* 


LPRINT  "MISSING  POINTS" 
LPRINT  "OTHER  POINTS" 


K.'y/j 


.V  o.v 


I 


.'•  s'* 

€ 

Ts* 

yc**? 

j*  ••  w 
L  V  % 


TREATMENT  OF  SAMPLE  NWS2AIX  USINQ  LS7T3.BAS 


OUTLIERS 


2136 

-12 

33466.5 

6 

4 

3 

33452.28 

14.22266 

4.610405 

2144 

-12 

33789.7 

6 

4 

3 

33812.96 

-14.25781 

2.9266.7 

2157 

-10 

33273.5 

5 

4 

2 

33238.63 

34.875 

2.541182 

MISSING  POINTS 
2120  -2 

33634. 1 

6 

1 

2 

33634.88 

-.7734375 

1.475461 

2128 

-2 

33810.0 

6 

3 

3 

33816.34 

-5.4375 

5. 16895 

2152 

—  O 

Jm 

33285. 1 

6 

3 

2 

33277.93 

7. 175782 

2. 356863 

2160 

-2 

33374.2 

5 

3 

3 

33369.4 

4.804688 

1.941253 

OTHER  POINTS 

2121 

0 

33664 . 6 

6 

1 

2 

33665.28 

-.6757813 

1.526676 

2122 

0 

33695 

6 

1 

2 

33693.54 

1 . 460938 

1 . 408332 

2123 

0 

33718.5 

6 

1 

2 

33720.87 

-2.371094 

2.412839 

2124 

0 

33745.8 

7 

1 

2 

33744. 58 

1 . 226563 

1 . 6*j>692*.i> 

2125 

0 

33767. 1 

6 

1 

2 

33764. 55 

2.550781 

3 . 020508 

2126 

0 

33780.7 

6 

1 

2 

33783.69 

-2.988281 

3. 133061 

2127 

0 

33798 

6 

1 

2 

33799.72 

-1.722656 

3.467283 

2129 

0 

33823.7 

6 

1 

3 

33828. 12 

-4.421875 

3.467283 

2130 

0 

33827.3 

6 

1 

3 

33821.64 

5.664063 

8.346241 

2131 

0 

33794.2 

6 

1 

2 

33786. 6 

7.605469 

1 1 . 22599 

2132 

0 

33726. 1 

7 

1 

3 

33724.05 

2.050781 

2.459811 

2133 

0 

33637 . 7 

6 

1 

3 

33640. 13 

-2.433594 

2.61768 

2134 

0 

33556.5 

6 

1 

o 

33556.91 

-.40625 

5.074126 

2135 

0 

33486.6 

6 

1 

33489.96 

-3.359375 

3.445967 

2137 

0 

33446.5 

6 

1 

2 

33451. 15 

-4.652344 

5. 162057 

2133 

0 

33485. 1 

6 

1 

33489.95 

-4.792969 

3.315254 

2139 

0 

•i>o559 .  o 

6 

1 

33560.92 

-1.617188 

3.443657 

2140 

0 

33650 . 2 

7 

1 

3 

33649.8 

.  •1*984375 

1.671507 

2141 

0 

o^i’>738 . 5 

6 

1 

33734.47 

4 . 03 1 2575 

5.  498794 

2142 

0 

33799 

6 

1 

3 

33795 . 56 

3.445313 

7.332694 

2143 

0 

33825.8 

6 

1 

o 

33823. 13 

2.671875 

2. 47908 

2145 

0 

33771 . 6 

6 

1 

•J* 

33767 . 36 

4.242188 

6.616293 

2146 

0 

33698 . 3 

6 

1 

3 

33696 . 5 

1.800781 

5.  109362 

2147 

0 

33607.7 

6 

1 

o 

33614.61 

-6.910157 

9.347524 

2148 

0 

33528.5 

7 

1 

3 

33529. 77 

-1.269531 

3.590484 

2149 

0 

33455.2 

6 

1 

o 

3345 1.71 

3.492188 

3.  333284 

2 1 50 

0 

33381.2 

6 

1 

•.!> 

33383 . 39 

-2. 1875 

2.737984 

2151 

0 

33323. 5 

6 

1 

33325. 17 

— 1 . 664063 

3.271413 

2153 

0 

33246.6 

6 

1 

33243.54 

3 . 066406 

3.0161 05 

2154 

0 

33219.5 

5 

1 

2 

33222. 07 

-2.570313 

4.597027 

2155 

0 

33212.5 

5 

1 

33214.04 

-1.542969 

4.636977 

2156 

0 

33221 

6 

1 

0*^2 18.3 

2. 199219 

2. 034624 

2158 

0 

33267.7 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  BASIC  program  LS7T3.BAS  presented  in  this  report  uses  the  Least- 
Squares  Method  to  select  a  polynomial  of  order  1,  2,  or  3  to  fit  7-point 
segments  of  data.  It  establishes  an  estimate  for  the  actual  value  at  the 
midpoint  of  the  data  segment  and  a  Figure  of  Merit  (FM)  to  provide  an 
indication  of  the  quality  of  that  estimate. 

A  modified  version  of  this  program  <LS7T4.BAS)  is  available  for  a  more 
detailed  examination  of  the  application  of  the  program  to  any  7-point  data 
segment . 

It  is  recommended  that  this  program  be  used  for  smoothing  NUWE5  3-D 
data.  It  is  also  recommended  that  this  program  be  transferred  to  a 
spreadsheet  format  for  use  with  LCTUS-123  with  the  intent  of  using  macros  to 
automats  its  application  to  the  extent  passible. 

Possible  modification  of  this  program  to  change  the  treatment  of 
multiple  questionable  points  in  a  data  segment  needs  further  examination  as 
suggested  in  Section  IV. 
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